The present invention relates to the field of video transmission, and more specifically to a system for live video streaming through a data or communications network to a client.
Live video streaming is a difficult technical challenge due to latency constrains, communications network bandwidth and system robustness. Such streaming gains exponential complexity once multiple points of view or channels require multiplexing. Examples are multiple Picture-in-Picture, Channel guide, and Multiuser Videoconferencing.
Low latency requirements and real-time content make the application of retransmission protocols, large jitter buffers and pause and play/buffering (trick play) techniques almost impossible. Complex communications network topology, with lack of control over intermediate nodes (e.g., the Internet), require a unidirectionally robust protocol with low latency built-in into the Client-Server (Application-Cloud) solution.
Adding to this complexity is the need to deliver multiple content streams to multiple clients with per-client stream multiplexing patterns. This typically calls for powerful video transcoding operations at the server such as, but not limited to, MCU (multiconferencing unit) in case of videoconferencing. The role of such operations is to compose a multiplexed image based on per-client requirements from the original full resolution streams arriving from multiple sources. This step allows the reduction of bandwidth and decoding requirements at the client which would otherwise be exposed to all the incoming full resolution streams.
For example, live concert streaming may present multiple camera views in the client application showing main/selected camera content in full resolution and additional cameras in low resolution preview mode for interactive selection or dynamic switching between views. Clicking on the preview channels switches to the full resolution stream from the selected source, while previous full resolution content falls back into preview mode.
Content switching may be accomplished manually, via user control, or automatically as in case of videoconferencing with multiple participants and full resolution content selection based on voice activity (e.g., active talker detection).
Another important property of live video streaming such as sporting events for example, is that interruption of the content flow has a more drastic impact on the user experience compared to loss of high resolution details. An example of this problem is a “freeze” of live streaming of a soccer match in a family living room during a critical shot, while all the neighbors are already screaming “goal!”
Therefore, what is needed is a new and improved way to deliver multiple content streams to multiple live video clients with per-client stream multiplexing patterns.